Multi-agent Coordination in Directed 
Moving Neighborhood Random Networks 

Yilun Shan^ 
Abstract 

In this paper, we consider the consensus problem of dynamical multiple agents 
. that communicate via a directed moving neighborhood random network. Each agent 

' performs random walk on a weighted directed network. Agents interact with each other 

, through random unidirectional information flow when they coincide in the underlying 

' network at a given instant. For such a framework, we present sufficient conditions for 

almost sure asymptotic consensus. Some existed consensus schemes are shown to be 
D ' reduced versions of the current model. 
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1. Introduction 

■ Synchronization and consensus problems in multi-agent systems have a long history 

M i [3]. Over the last few years, considerable research attention has been paid to cooperative 

^\ ' group coordination, which has broad applications in a variety of areas including vehicle 

^ ■ formation, distributed sensor networks, coupled oscillator systems, animal aggregation 

I— I. and epidemics; see the survey papers (lOl HH EH] and references therein. In a consensus 

^ I problem, a group of autonomous agents seeks to agree on some quantities of interest 

^ ■ via a process of distributed decision making based upon local information interactions. 

: A consensus is reached when all agents in the system tend to attain agreement on the 

I quantities of interest as time evolves, that is, they converge to a common value. 

' The ability to achieve coherent behavior (or consensus) of multi-agent system is inti- 

Q\ • mately related with the coupling topology of the underlying communication graph associ- 

O I ated with the agents' positions. For example, Ali Jadbabaie et al. analyzed a simplified 

model of flocking by Vicsek et al. [20] showing that all agents will synchronize eventually, 
provided the communication graph switching deterministically over time is periodically 
i jointly connected. More recently, there has been some interest in stochastic consensus 

^ ' problems. [1] deals with the agreement problem among a group of static agents with the 

' communication graph presented as Erdos-Renyi random graph. The results are further 

extended by [12] to include weighted and directed information flows. In [TS], the authors 
reveal this problem can be reduced to weak ergodicity of a sequence of stochastic matrices. 
However, the agents are static in these works, in other words, the switching of communi- 
cation graph is merely due to the adjunctive random mechanism rather than the motion 
of agents. 

The moving neighborhood network proposed in [16j is meant to treat stochastic con- 
sensus among dynamical agents (in social interaction and epidemics originally), where 
each agent carries an oscillator and diffuses in the environment. The computer simula- 
tion in the aforementioned paper shows that synchronization is possible even when the 
communication network is spatially disconnected in general at any frozen time instant. 
This is in sharp contrast with the behavior of Vicsek model mentioned above. Analytical 
formalizations are provided by authors of [iTl [HI [9] with different consensus protocols 
(continuous-time with dwell time or discrete-time) through fast switching techniques |17j 
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and stochastic stability theory [7]. In particular, they specify the moving neighborhood 
network model as a group of n identical agents implementing simple random walks on a 
fixed finite connected graph. The vertices of the moving neighborhood network are rep- 
resented by the agents, and the edges are determined by their locations in the underlying 
graph, i.e. a link between two agents appears if and only if they reside in the same node 
simultaneously. The time evolution of the moving neighborhood graph governed by the 
random walk of agents is called network dynamics and the time evolution of the corre- 
sponding oscillators system is called system dynamics, which is coupled together with the 
network dynamics. 

The purpose of this note is to generalize the above results and go a further step in 
the direction of [9j. Our extension is threefold. Firstly, we consider the underlying fixed 
graph as a weighted directed graph. In other words, random walks on weighted directed 
graph is tackled. Secondly, further randomness is involved in the moving neighborhood 
graph. A probability is associated (not necessarily independent) to the link between two 
agents when they occupy the same node, which can be interpreted as communication links 
may be unreliable due to disturbance. We allow different probabilities and unidirectional 
communications. Therefore the topology of the moving neighborhood graph is no longer 
strictly separated clique-like. Finally, we study a discrete-time consensus protocol with 
inhomogeneous weights, which quantify the way the agents influence each other. By these 
modifications, some pertinent known results can be recovered (see the remarks in Section 
2 &; 4) , and a number of introduced parameters enhance more flexibility. 

The rest of this paper is organized as follows. In Section 2, we elaborate on our formal 
model under consideration. Section 3 contains some mathematical preliminaries and the 
statement of main result. Proofs are given in Section 4. 

2. Problem setup 

Let G = (y,E,W) be a weighted digraph with vertex set V = [m], here [m] := 
{1, 2, • • • , m}. The elements in V will be referred to as nodes. £^ is a set of ordered pair 
of elements of V called arcs. W = {wij) is the weight matrix, where Wij > if (i, j) G E, 
and Wij = otherwise. Notice that we do not exclude loops. We impose the following 
condition on the graph G. 

Assumption 1. The underlying graph G is strongly connected and the gcd (greatest 
common divisor) of all cycle lengths in (5 is 1. 

This assumption will be elucidated in the next section. Let t G NU {0} be the discrete 
time step. We consider n identical agents signified by {vi,V2-, • • • , fn} as random walkers 
meandering on G, moving randomly to a neighbor of their current location in G at each 
time step. For each agent, the random neighbor that is chosen is not affected by the 
agent's previous trajectory. The n random walk processes are mutually independent. A 
time- varying moving neighborhood graph, G = {V,E,P), is constructed as follows. Let 
V = V(t) = {vi,V2, • • • , Vn} be the vertex set. P = (pij) is an n x n matrix with entries 
< Pij < 1 for i ^ j, and pa = for 1 < i < n. P is dubbed as the linkage probability 
matrix. Given Vi and Vj meet at the same node simultaneously, an arc {vi,Vj) originating 
from Vi and terminating in Vj appears with probability pij. The arc set is denoted by 
E = E{t). We emphasize here that the selections of arcs are independent with the random 
walk processes, but are not required to be independent with each other. By definition, 
G = G{t) has no loops with probability 1. 

Let Xi{t) G M be the state (attitude, heading, opinion etc.) of agent Vi at time t. The 
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consensus protocol can be expressed as 



Xi{t + l)=Xi{t) + e hj{X,{t) - Xiit)) 



(1) 



ieAf,(t) 



where Ni{t) is the index set of neighbors of agent Vi at instant t, i.e. Ni{t) := Vj) G 

-E(t)}. Here, weighting factor bij > for i 7^ j, and ba = for 1 < i < n. Let 
A{t) = {aij{t)) be the adjacency matrix of the moving neighborhood graph G'(t), whose 
entries are given by, 



The imphcation of Assumption 2 is the out-degrees are equal to in-degrees for every 

V G V{t). A similar definition used in [11] is referred to as "balanced " digraph. Note that 
in the context of [11], the underlying communication network is static, while in the current 
case, we dictate each trajectory of G{t) should be "balanced". However, this requirement 
is not very stringent after all. For example, if we postulate G{t) can only have bidirectional 
(or undirected) edges, that is, bij = bji and pij = pji for 1 < i, j < n, then Assumption 
2 is satisfied automatically. Also, if bij = b for i ^ j and suppose for each v G V{t), the 
numbers of arcs going out and coming in are equal, then Assumption 2 is filled, too. It is 
worth noting that since the selections of arcs can be arbitrarily dependent with each other 
as mentioned before, the latter of the two examples above can actually occur. 

Next, let A := maxi<j<„(^"^]^ and we further assume e G (0, 1/A). This as- 
sumption is natural; see Section 4. 

As usual, denote the linear subspace ^A := {x = {xi,X2, • • • , x„)^ G M"| Xi = Xj y 1 < 
hj ^ n} as the synchronization manifold. A consensus is said to be reached if the states 
of agents converge into Ai as t approaches infinity. We aim to show that the n agents in 

V reach consensus almost surely. 

We end this section by two remarks. 

Remark 1. Suppose that the linkage probabilities Pij{t) and the weighting factors bij{t) 
are time-dependent, then the main result in this paper holds verbatim as long as Pij{t) 
and bij{t) converge. 

Remark 2. If we take the underlying graph G to be a single node, then our framework 
somewhat reduces to that of [U [181 IS] ■ 

3. Preliminaries and main result 

Let {Yi{t),t G N U 0} denote the random walk process performed by agent Vi such 
that Yi{t) G V designates the position of Vi in the underlying graph G at instant t. For 
1 < i < n, let -7Ti{t) = {iTii{t), ■ ■ ■ ,TTim{t))'^ be the probability distribution of Yi{t) at time 
t, that is, P{Yi{t) = j) = 7rij{t) for j G V. By the construction, {Yi{t),l < i < n} are 
n independent homogeneous Markov chains with finite state space V, sharing the same 
transition probability matrix Q = (qij), whose entry is given by 




bij, {vi,Vj) G E{t) 
0, otherwise 



for 1 < i, j < n. Moreover, we assume the following 



Assumption 2. Suppose that Yl o-ijit) = S for 1 < i < n. 

i=i j=i 




for 1 < i,j < m 



(2) 
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where di = X^j^i Wij represents the out-degree of node i. Also recall that qij = P{Yi{t + 
1) = j\ Yi{t) = i). Notice that by Assumption 1 7^ 0, so ([2]) is well-defined. 

As is known, a finite Markov chain converges to a unique stationary distribution if it 
is ergodic, i.e. irreducible and aperiodic (see e.g. [15] Thm. 4.2). It is clear that the 
transition probability matrix Q is irreducible if and only if G is strongly connected. Also, 
it's easy to see that the gcd of all cycle lengths in G is 1 if and only if all eigenvalues of 
Q other than 1 have modules strictly less than 1. Actually, the above assertions may be 
proven by employing Perron-Frobenius Theorem [15] and some other equivalent conditions 
in terms of the ergodicity of random walk are given in [2]. Consequently, in view of 
Assumption 1, Yi{t) converges to a unique stationary distribution vr = (tti, • • • ,7rm)'^ for 
1 < i < n, that is, vr-^ = tt'^Q. For instance, if X^jLi Wij = YlY=i for 1 < i < m, then 
we have tt, = di/Y^^j=i "^ij for 1 < i < m, see e.g. [8]. As for the rate of convergence, we 
have the following lemma, (see [1] Thm. 8.9 for a proof). 

Lemma 1.(11]) For a finite ergodic Markov chain with transition probability matrix P, 
let p^j^ be the {j,i) entry of P^, f G N U 0, then there is a unique stationary distribution 
{pi} and 

\pf^ -Pi\<Xp\ 

where A>0, 0<p<l. 

Next, let D{t) = diag((iii(t), • • • ,dnn{t)) be the out-degree diagonal matrix of the 
moving neighborhood graph G{t). To be precise, dii{t) = X]j'=i for 1 < z < n, 

where aij{t) are the entries of A(t) defined in Section 2. Notice that A{t) and D{t) are 
non-negative time-dependent random matrices. Since aij{t) is a two-point distribution, its 
expectation is shown to be given by 

m 

E{aij{t)) = bijPij^7rik{t)7rjk{t), for i ^ j, (3) 
k=l 

and aii{t) = 0. The diagonal entries of D(t) can be written as 

m n 

diiit) = '^ ^ bijl[Yi{t)=k, Yj(t)=k, (v,,Vj)eE(t)]- 

k=lj=l,j7^i 

Thereby, E{dii{t)) = Y.T=i YJj=i hjPij'^ik{t)T^jk{t), for 1 < f < n. Now let L{t) = D{t) - 
A{t) denote the Laplacian matrix [2j of G{t), and it's expectation at time t is EL{t) = 
ED{t)-EA{t). 

For a sequence of random elements Z{t), the ergodic limit, E*{Z), is defined by 
E*{Z) = Imit^oo E{Z{t)), whenever the limit exists. E*{Z) describes the long-run av- 
erage respect to its stationary distribution [13]. By the above discussion, the ergodic limit 
of © is E*{aij) — bijPij Yl^=i "^k ^ 7^ J ^J^d E*{aii) — for 1 < i < n. Likewise, 
E*{dii) = XlfcLi ^fc YTj=i bijPij for 1 < i < n. 

Now we make a reference to Schur product of two real matrices, which is also known 
as Hadamard product [3|. Let C = (cij), E = {eij) be two n x m real matrices, then the 
Schur product, C o E = (cijCij), is simply the product of corresponding entries of C and 
E. Obviously, C o E = E o C . Utilizing this notation, we obtain E* (A) = tt^ttB o P and 
E*{D) = vr^7rdiag(^"^i hjPij, ■■■ , =1 bnjPnj)- Hereby, we have 

n n 

E*{L) = 7r'^7r(diag( bi.pij, ■ ■ ■ ,J2 KjPnj) - B o p) . (4) 
i=i 3=1 
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E*{L) is positive semi-definite and have an eigenvalue since E*{L)1 = 0, where 1 is the 
aU-1 column vector. We need the following lemma regarding Schur product. 

Lemma 2. Suppose C, E are two nxn real matrices and x, y are two nxl real vectors, 
then 

y^{C o E)x = tr(diag(?/^)^Cdiag(2;^)E^). 

The proof will be given in the next section. An asymptotic stability result in [7] is 
restated below for convenience, which is an analogy of the deterministic Lyapunov stability 
theorem. 

Lemma 3.(^, pp.195) Let {Xn} be a Markov chain on state space S. Suppose that 
there is a non-negative function (x) satisfying 

E{C{Xi)\Xo = x)-C{x) = -7]{x), 

where rj{x) > on the set Qp := {x : S,{x) < (3]. Then 

P(supe(X„) > /3|Xo = x)< 

n>0 

and accordingly, f]{Xn) almost surely for paths which remain in Q^. 

It is at this stage, we state our main result as follows. 

Theorem 1. Under the circumstances and assumptions presented above, the stochastic 
system expressed by (QP reaches consensus almost surely. 

4. Proofs 

This section includes the proofs of Lemma 2 and Theorem 1. 
Proof of Lemma 2. Let 1 = (1, 1, • • • , 1)'^ be the nxl vector. Note that diag(y^)l = y. 
Therefore, we get 

y^{CoE)x = l^diag{y^f{C o E)x 
= l^{diag{y^fCoE)x 
= tv{diag{y^fCdiag{x^)E^) 

We have exploited a basic property: {Adiag{x^)B^){i,i) = {{A o B)x){i) in the last 
equality above. Here notation A{i,i) means the entry of matrix A, and x{i) the ith 
element of vector x. □ 

Proof of Theorem 1. We rewrite the protocol in a compact matrix form as 

X{t + l) = F{t)X{t), (5) 

where X{t) = • • • is the state vector of agents at instant t, and F{t) = 

In - eL{t). The spectrum of F{t) satisfies 1 = A„(F(t)) > • • • > Xi{F{t)) > 1 - 2eA, since 
the eigenvalues of Lapacian L{t) are 2A > Xn{L{t)) > • • • > Ai(L(t)) = 0; see [2] for more 
about Laplacian spectrum. 

Recall we assume e £ (0, 1/A) in Section 2, which yields \Xi{F{t))\ < 1 for 1 < i < n 
and causes the state transition matrix F(t) to be stable at each time step, (see e.g. [10] 
Lemma 3). By the Assumption 2 and iterative equation ([5]), we obtain 

l^X{t + 1) = l^F{t)X{t) = {F{t)X{t)fl = X{tfF{tfl = {X{tflf = l^X{t). 
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Prom this, it is clear that the projection of the state vector X{t) on synchronization 
manifold is a constant. In fact, let a := ^ SILi -^ii^)^ then l^X{t) = na for all t. 
Now we may decompose X{t) as 

X{t)=X'P^{t)+XP\t), (6) 

where X'P"-{t) G M and X'P^{t)^^M. The superscript "pa" stands for "parallel", while 
"pe" for "perpendicular". We thereby get = ■^\^X{t) = y/na. Here and in 

the sequel, we take || • || as 2-norm for vectors and induced 2-norm for matrices [5]. The 
disagreement among the agents now can be described by = — na^. To 

prove the theorem, it suffices to show — >■ almost surely as t ^ oo. 
We follow [9] to introduce a Lyapunov function ^(x) = x'^x — a^n, and let 

i{t) ■= i{X{t)) = X{tfX(t) - a^n. 

Our plan is to use Lemma 3 to show ^(t) almost surely, as t — > oo. Note that X{t) is 
indeed a Markov chain. With this in mind, we evaluate by employing ([5]), 

E{i{t + l)\X{t)) = X{tfE{F{tfF{t))X{t) - a^n. 

Therefore we get 

Ei^t + l)\X{t)) - m = X{tfEH{t)X{t) (7) 

where, H{t) := F{t)'^F{t) -1 = eH{tfL{t) - e{L{tf + L{t)). 

Let Li be the ith possible realization of with probability Pi{t) at instant t, and 
denote VL as the collection of all Li. We hereby may write the expectation of L{t) as 
EL{t) = Y^i^i^iPii^)- 1^1 the cardinality of O. Since L{t) inherited the property of 
the random walks of the agents, it is also an ergodic Markov chain with state space 0. 
Let Pi be the ith. component of the unique stationary probability distribution. Then we 
may apply Lemma 1 and interpret the corresponding meaning of notations in the current 
situation. By doing so, we derive 



\pi{t) - = I Y^Pji^M -Pr = \ Y^^Pji - Pi)pM 

\n\ \n\ 
< 5^|pJ?-p.h(0)<A/.*j;p,(0) = Ap*. (8) 

Now going back to the equation ([Tj), we have 

\n\ 

XitfEHit)Xit) = Y,p,{t)X{tf HiX{t) (9) 

i=l 

where Hi = e^Lj Li — e{Lj + Li) is the ith possible realization of H{t) and is negative 
semi-definite with eigenvalues = Xn{Hi) > • • • > Xi{Hi). Using the decomposition ([6]), 
we get 

X{tfHiX{t) = XP%tfHiXP%t) + XP''{tfHiXP%t) 
+XP%tfHiXP^{t) + XP^{tfH^XP^{t) 
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There are four terms on the right hand side of the above equation, the last three of which 
are actually zero. To see why let's take the second term as an example, 

XP^iifHiXP^t) = e^XP''itfLjLiXP%t) - eXP^itf lJ XP^t) - eXP^itf UXP^t) 
= e^{LiXP''{t)fLiXP^{t) - e{LiXP^{t)fXP^{t) 

-e{LfXP^{t)fXP''{t) 
= 

since UXP'^it) = ifXP'^it) = 0. Thereby we have X{tfHiX{t) = XP%t)^ HiXP%t) . 
Plugging this equation into Q, we get 

X{tfEH{t)X{t) = XP^{tf(^p,{t)Hi)xP%t) = XP''{tfEH{t)XP%t). 

1=1 

We have the ergodic limit, defined in Section 3, of H{t) as 

E*H = lim EH{t) = lim E{e^L(tfL{t) - e{L{tf + L{t))) 



e^E* (L^L) - e{E*L^ + E*L) (10) 



since we know L{t) is an ergodic Markov chain. 

Take R{t) := EH{t) — E*H as the remainder matrix, then we have 

XP^{tfEH{t)XP^{t) = XP^{tf{E*H)XP%t) + XP%tfR{t)XP%t). (11) 

The second term on the right hand side of ([TT]) can be calculated, by noting dSj), as follows 



\n\ 

XP%tfR{t)XP%t) = \\XP%tfRit)XP%t)\\ = ||XP^(t)^(^//,(p,(t) -p, 

1=1 

\n\ 

^ \\XP'it)\\' ■\\Y.H,{p,it)-pi 
1=1 

< amh^p', (12) 

where 7 := maxi<j<|Q| Wherefore, (fT2]) tends to 0, since < p < 1 and ^{t) is 

bounded by ([T|). 

Next, we turn to the first term on the RHS of pi|) . In view of pU|) . 

XP^{tf{E*H)XP%t) = e^XP%tf{E*L^){E*L)XP%t)-eXP%tf{E*L'^)XP<'{t) 

-eXP\tf{E*L)XP^{t). (13) 

By using Lemma 2, we have 

XP^{tY{B'^ oP'^)XP\t) = tT{diag{XP^(tf)B^diag{XP''{tf)P) 

= ti{dmg{XP^{tf)Bdiag{XP^{tf)P^) 
= XP%tf{BoP)XP%t). 
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Then it's easy to see the last two terms on the RHS of (|13p is equal, by noting equation 
([3]). Thereby we obtain 

XP%tf{E*H)XP^{t) = e^XP%tf{E*Lf{E*L)XP%t)-2eXP%tf{E*L)XP%t) 

< (2Ae2 - 2e)XP%tf{E*L)XP%t) 
= 2e7r^7r(eA - 

n n 

•(diag( hjPij, ■■■ ,Y1 ^-^Pr^^) - ^ ° P)X''\t) 
3=1 i=i 

< 2e^^7rA(;rA - l)^(t) (14) 

The second inequality above holds because there is some N{B) > depends only on B, 
such that {E*L)'^{E*L) < 2A{E* L) + N{B)ll'^ and 

XP^(tfN{B)ll'^XP%t) = N{B){l^XP^{t)f{l^XP%t)) = 0. 

The fourth inequality holds by noting that eA — 1 < and 

n n 

(diag( hjPij, ■■■ ^^^P^j) -BoP)>AI- M{B)11^ 
i=i j=i 

for some M{B) > depending only on B. The similar argument as above may be applied. 
Combing ([12]) and (fH|) . we finally get 



E{i{t + l)\X{t))-i{t) < C(t)(|r?|7Ap* + 2e7r^^(eA - 1)A) 

:= mm--=-ri{t) 

There exists to G N such that 5{t) < for t > to- Furthermore, 5{t) 2e7r'^7r(eA - 1)A, 
as t — > CO. Also, note that ^(t) is a Markov chain, since X[t) is. By applying Lemma 3, 
where (5 can take any positive number, we obtain 

'q{t) ^0 as t ^ CO 

almost surely, which yields ^(t) almost surely, as t — > oo. The proof now is completed. 

□ 

Remark 3. Theorem 1 shows an average consensus which may be regarded as a kind of 
stochastic version of Theorem 9 in |llj . 
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